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Abstract — In this paper, a novel receiver for Hexagonal Mul- 
ticarrier Transmission (HMT) system based on the maximizing 
Signal-to-Interference-plus-Noise Ratio (Max-SINR) criterion is 
proposed. Theoretical analysis shows that the prototype pulse 
of the proposed Max-SINR receiver should adapt to the root 
mean square (RMS) delay spread of the doubly dispersive 
(DD) channel with exponential power delay profile and U-shape 
Doppler spectrum. Simulation results show that the proposed 
Max-SINR receiver outperforms traditional projection scheme 
and obtains an approximation to the theoretical upper bound 
SINR performance within the full range of channel spread factor. 
Meanwhile, the SINR performance of the proposed prototype 
pulse is robust to the estimation error between the estimated 
value and the real value of time delay spread. 

I. Introduction 

Orthogonal frequency division multiplexing (OFDM) sys- 
tems with guard-time interval or cyclic prefix can prevent 
inter-symbol interference (ISI). OFDM has overlapping spec- 
tra and rectangular impulse responses. Consequently, each 
OFDM sub-channel exhibits a sinc-shape frequency response. 
Therefore, the time variations of the channel during one 
OFDM symbol duration destroys the orthogonality of different 
subcarriers, and results in power leakage among subcarriers, 
known as inter-carrier interference (ICI), which causes degra- 
dation in system performance. In order to overcome the above 
drawbacks of OFDM system, several pulse-shaping OFDM 
systems were proposed |r|-|4|. 

It is shown that signal transmission through a rectangular 
lattice is suboptimal for doubly dispersive (DD) channel [51- 
|U|. By using results from sphere covering theory, the au- 
thors have demonstrated that lattice OFDM (LOFDM) system, 
which is OFDM system based on hexagonal-type lattice, 
providing better performance against ISI/ICI 15|. However, 
LOFDM confines the transmission pulses to an orthogonal set. 
As pointed out in [2], these orthogonalized pulses destroy the 
time-frequency (T-F) concentration of the initial pulses, hence 
lower the robustness to the time and frequency dispersion 
caused by the DD propagation channel. 

In lO-im, the authors abandoned the orthogonality con- 
dition of the modulated pulses and proposed a multicar- 
rier transmission scheme on hexagonal T-F lattice named 
as hexagonal multicarrier transmission (HMT). To optimally 
combat the impact of the DD propagation channels, the lattice 
parameters and the pulse shape of modulation waveform are 



jointly optimized to adapt to the channel scattering function 
from a minimum energy perturbation point of view. It is 
shown that the hexagonal multicarrier transmission systems 
obtain lower energy perturbation by incorporation the best 
T-F localized Gaussian pulses as the elementary modulation 
waveform, hence outperform OFDM and LOFDM systems 
from the robustness against channel dispersion point of view 

m-M. 

In HMT system, there is no cyclic prefix and the data 
symbols of HMT signal are transmitted at the hexagonal lattice 
point of T-F plane and subcarriers are interleaved in the T- 
F space, as is illustrated in Fig. I. The basic mathematical 
operation on the received signal performed by the demodulator 
is a projection onto an identically structured function set 
generated by the prototype pulse function llT0l - lfT2l . i.e. an 
optimal match filter. It is shown in [T3l-fT5l that the optimum 
sampling time of wireless communication systems over DD 
channel depends on the power distribution of the channel 
profiles, and that zero timing offset does not always yield the 
best system performance. Max-SINR ISI/ICI-Shaping receiver 
for multicarrier modulation system is discussed in the 
Max-SINR prototype pulse can be obtained by maximizing 
the generalized Rayleigh quotient. 

We will present in this paper that receivers proposed in 
f6l-f8l are suboptimal approaches in the view of Signal-to- 
Interference-plus-Noise Ratio (SINR) performance. A novel 
receiver based on maximizing SINR (Max-SINR) criterion for 
HMT system is proposed in this paper Theoretical analyses 
and simulation results show that the proposed optimal Max- 
SINR receiver outperforms the traditional projection receiver 
in fS'l-fSl and obtains an approximation to the theoretical 
upper bound SINR performance. Meanwhile, the proposed 
scheme is robust to the estimation error between the estimated 
value and the real value of root mean square (RMS). 

II. Hexagonal Multicarrier Transmission System 

In the view of signal transmission on lattice in the T-F 
plane, the system performance is mainly determined by two 
factors: a) energy concentration of the elementary modulation 
pulse, a better T-F concentrated pulse would lead to more 
robustness against the energy leakage and b) distance between 
the transmitted symbols in the T-F plane, it is obvious that 
the larger the distance, the less the perturbation among the 
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Fig. 1. Partition of the liexagonal lattice into a rectangular sublattice Vi-ccti 
(denote by o) and its coset Vicct2 (denote by •). 



transmitted symbols. As pointed out in |l6l-|l8l, for a given 
signaling efficiency, the information-bearing pulses arranged 
on a hexagonal T-F lattice, as is illustrated in Fig. 1, can be 
separated as sufficiently as possible in the T-F plane. 

In HMT systems, the transmitted baseband signal can be 
expressed as fS) 



mT - 



J27r(nF+f )t 



(1) 



where T and F are the lattice parameters, which can be 
viewed as the symbol period and the subcarrier separation, 
respectively; Cm,n is the user data, which is assumed to be 
taken from a specific signal constellation and independent 
and identically distributed (i.i.d.) with zero mean and average 
power a"^; m G Ai and 7i G Af are the position index in the 
T-F plane; A4 and J\f denote the sets from which m, n can be 
taken, with cardinalities M and A^, respectively. The prototype 
pulse g{t) is the Gaussian window 



g{t) = (2/a)i/4e-("/'^)*' 



(2) 



with (7 being a parameter controlling the energy distribution 
in the time and frequency directions. The ambiguity function 
of Gaussian pulse is defined by 



Ag{T,v) 



(3) 



where (•)* denotes the complex conjugate. 

It is shown in Fig. 1 that the original hexagonal lattice can 
be expressed as the disjoint union of a rectangular sublattice 



Viecti and its coset yiect2- The transmitted baseband signal in 
(1) can be rewritten as 



(4) 



where i=l,2, „ and „ represent the symbols coming 
from two sets Vrecti and yrect2, respectively. gln,n{t) = g{t — 
mT — ^)e''^^'-"^"'"^-'* is the transmitted pulse on m-th 
symbol and rt-th subcarrier 

The baseband DD channel can be modeled as a random 
linear operator H lfT6l 



U[x{t)] 



fd 



H{t, v)x{t - T)e^^-^''^dTdv (5) 



— fd 



where Tmax and fd are the maximum multipath delay spread 
and the maximum Doppler frequency, respectively L17i . 
H{t, v) is called the delay-Doppler spread function, which 
is the Fourier transform of the time- varying channel impulse 
response h{t,T) with respect to t. The product d = T^ax/d is 
referred to as the channel spread factor (CSF). If ?9 < 1, the 
channel is said to be underspread; otherwise, it is overspread. 
Practical wireless channels usually satisfy the assumption of 
wide-sense stationary uncorrected scattering (WSSUS), and 

7?< 1 ma. 

In the WSSUS assumption the channel is characterized by 
the second order statistics 

E[H{T,v)H*iTl,Vl)] = SH{T,v)SiT ^ Tl)S{v ^ Vl) (6) 

where E[-] denotes the expectation and Sh{t,v) is called 
the scattering function, which characterizes the statistics of 
the WSSUS channel. Without loss of generality, we use 
Jo"'" J-f^ Snir, v)dTdv = 1. 

As shown in (5), the propagation channel introduces energy 
perturbation among the transmitted symbols. It is shown 
in L5J-|i7J that the symbol energy perturbation function is 
dependent on the channel scattering function and the pulse 
shape. The properly designed HMT system with Gaussian 
prototype pulse achieves minimum symbol energy perturbation 
over DD channel. The optimality of such system in combating 
the ISI/ICI caused by the DD channel is guaranteed by the 
Heisenberg uncertainty principle and the sphere-packing the- 
ory. The choice of the system parameters (cr, T, F) to minimize 
such undesired energy perturbation is extensively dealt with in 
|5|-|7|. In general, the pulse shape of the Gaussian window 
(J and the transmission pattern parameters T and F should be 
matched to the channel scattering function. The optimal system 
parameter for DD channels with exponential-U scattering 
function can be chosen as |601 



fd 



a— r- — 



(7) 



'in the case of no time and frequency dispersion, SniT, v) = 5{t)S{v), 
the matching criteriaes (7) and (8) are also reduced to {^/3T) / (F) = 
{Wt)/{Wf) and T/{V3F) = {Wt)/{Wf), respectively. and W'^ are 
the centralized temporal and spectral second-order moments, respectively 1171 . 



TABLE I 

Factor a for Various Signaling Efficiencies 



where the interference-plus-noise energy 
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0.5 


1.0 


2.0 


4.0 
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2.25 


2.00 


1.90 


1.85 



and 



'^rms \ T 

17 " V^F 



(8) 



where Trms is the RMS delay of the DD channel and the 
coefficient a for various signaling efficiency p is listed in Table 
I. The received signal can be expressed as 

r{t) ^H[x{t)]+w{t) (9) 

where w{t) is the AWGN with variance cr^. 

III. Max-SINR Receiver for HMT System 

The basic mathematical operation of the received signal per- 
formed by the demodulator is a projection onto an identically 
structured function set generated by the prototype pulse func- 
tion, i.e. an optimal match filter [7J, [llj. To obtain the data 
symbol „, the match filter receiver projects the received 
signal r{t) on the prototype pulse function set V'm n(i)j *=1: 2, 
i.e., 

= E E ^L',n'{ii[9L',n'it)lrm,um (10) 
j m' .n' 

where V'm,„(i)=V'(i - mT - iT)eJ'2'^("-P'+"f )*, and ij{t) is 
the prototype pulse at the receiver The energy of the received 
symbol, after projection on the prototype pulse set i/'*„„(i) 
can be expressed as 



(11) 



Under the assumptions of WSSUS channel and source symbols 
are statistically independent, (11) can be rewritten as 



E, = a', / / SHiT,v) 

J T J V 



(\Ag,^{mT + T,nF + v)\ 



(12) 



T F , 

\Ag,^,{mT+-+T, nF+-+v)\ 



drdv 



The SINR of received signal can be expressed as 



■RsiN = ^ / / SH{T,v)\Ag^^^{T,v)\^dTdv 



(13) 



-Bin ^ I I Sh{t,v) 



^ {\Ag^^{mT + T,nF + v)\ 

z = [m,n]^#[0,0]^ ^ (14) 



T F ,2 

\Ag^,f, {mT +-+T,nF+-+v)\ 



drdv 



+ al\Ag.^^{0,0)\ 



Clearly, the interference-plus-noise energy function i?iN is 
dependent on the channel scattering function and the pulse 
shape (through its ambiguity function). According to the form 
of channel scattering functions, the Max-SINR receiver can be 
discussed in two cases fS'l: Case A: DD channel with exponen- 
tial power delay profile and U-shape Doppler spectrum. Case 
B: DD channel with uniform power delay profile and uniform 
Doppler spectrum. Due to the space limitations, this paper 
only to discuss the Case A, that is the Max-SINR receiver for 
HMT system over DD channel with exponential power delay 
profile and U-shape Doppler spectrum. 

For the DD channel with exponential power delay profile 
and U-shape Doppler spectrum, the scattering function can be 
expressed as IITSl 



(15) 



with r > 0, < fd- We assume thatH V'(i) = g{t-At), the 
theoretical SINR of the received signal over the DD channel 
with exponential power delay profile and U-shape Doppler 
spectrum can be expressed as 



sin 



7rTi-m.s/dB^IN Jo 



zdv 



(16) 



Substituting (15) into (14), the interference-plus-noise energy 



^It is shown in (_13J-[151 that the optimum sampling time of wireless 
communication systems over DD channel depends on the power distribution 
of the channel profiles, and that zero timing offset does not always yield the 
best system perfonnance. In other words, there is a timing offset between 
the prototype pulses at the transmitter and the receiver In |2|. the Max-SINR 
prototype pulse g of multicanier transmission system with rectangular' T-F 
lattice over DD channel is obtained by maximizing the generalized Rayleigh 
quotient g = arg max ■ The solution is the generalized eigenvector of 
the matrix pair {B,A) corresponding to the largest generalized eigenvalue. 
It is shown that there is a delay between the transmitted Gaussian prototype 
pulse and the received Gaussian prototype pulse. In this paper, the close form 
time offset expressions between the transmitted and received prototype pulse 
is derived. 



function En^ in (16) can be expressed as 



ElN — 



E 



fd ^-aTr{7iF+v)^ 



2_ TrCmT + T-At)" 

e ^rms e " dr 



zdv 



POO 

(m,n)5^(0,0) ° 



'iT+-i-+T-At I 



-dv ^ +<|Ag,v(0'0)| 



(17) 



The theoretical SINR upper bound of the received signal 
can be expressed as 



i?uB = argmaxi?siN 

At 



(18) 



Plugging (16) and (17) in (18), the Max-SINR prototype pulse 
can be expressed as ip^t) = g{t — At) and (see Appendix) 

At = 



2nT,T 



2tt [ 1.76 j 

We can see from equation (19) that the prototype pulse 
of the proposed Max-SINR receiver is a function of RMS. 
In HMT system, the prototype pulse shape of the Gaussian 
window and the transmission pattern parameters T and F 
should be matched to the channel scattering function including 
RMS. In other word, the RMS delay spread and the maximum 
Doppler frequency are known at the HMT transceiver Hence, 
it is reasonable to assume that the RMS delay spread priori 
known in HMT system. 

IV. Simulation and Discussion 

In this section, we test the proposed Max-SINR receiver via 
computer simulations based on the discrete signal model. In 
the following simulations, the number of subcarriers for HMT 
system is chosen to iV=40, and the length of prototype pulse 
A^g=600. The center carrier frequency is /c=5GHz and the 
sampling interval is set to Ts=10~®s. The system parameters 
of HMT system are F=25kHz, T=l x lO^'^s and a is set to 
a=T/^F. Traditional projection receiver proposed in (0- 
m is named as Traditional Projection Receiver (TPR) in the 
following simulation results. WSSUS channel is choosen as 
DD channel with exponential power delay profile and U-shape 
Doppler spectrum. 

The prototype pulses of Max-SINR receiver at i9=0.1 and 
i?=0.04 are given in Fig. 2. The prototype pulse of TPR scheme 
is also depicted for comparison. We can see from Fig. 2 that 
there is a delay between the TPR prototype pulse and the 
Max-SINR prototype pulse. The delay is obtained according 
to equation (19), and increases with the increasing of CSF i9. 
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Fig. 2. The compaiison of prototype pulses for different receivers at i9=0. 1 
and i?=0.04, respectively. 



The SINR performance of different receivers with the vari- 
ety of (Tc/'^'u) for HMT system over DD channel is depicted 
in Fig. 3. The CSFs are set to ??=0.07 and z9=0.2, respectively. 
We can see from Fig. 3 that the SINR performance of the 
proposed Max-SINR receiver outperforms TPR scheme about 
1- 4dB at i?=0.07 and 1.5~ 3.5dB at i?=0.2, respectively. The 
SINR gap between the proposed Max-SINR receiver and the 
theoretical SINR upper bound is smaller than 0.5dB and 0. IdB 
at z?=0.07, 0.2, respectively. 

The SINR performance of the proposed Max-SINR receiver 
with the variety of crl/a'^ is given in Fig. 4. We assume 
that there is an estimation error between the estimated value 
of RMS delay spread Tj^^^ and the real value Trms. We can 
conclude from Fig. 4 that the proposed Max-SINR receiver is 
robust to the estimation error of RMS delay spread. 

The SINR performance with the variety of channel spread 
factors -d at al/(jl,=2QdR is depicted in Fig. 5. The SINR 
performance of the TPR scheme is depicted for comparison. 
It can be seen that there is a degradation of SINR with the 
increasing of channel spread factor The proposed Max-SINR 
receiver obtains an approximation to the theoretical upper 
bound SINR performance within the full range of d. There is 
an about 2.5dB maximum SINR gap between the Max-SINR 
receiver and the TPR scheme at t?=0.35, and the SINR gap 
decreases as the CSF d decreases. 

V. Conclusion 

The Max-SINR receiver is proposed for HMT system over 
DD channel in this paper After a detailed analysis, we present 
that the prototype pulse of the Max-SINR receiver should 
adapt to the RMS delay spread. Theoretical analysis shows that 
the proposed Max-SINR receiver outperforms the TPR scheme 
in SINR and obtains an approximation to the theoretical 
upper bound SINR performance within the full range of CSF. 
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Fig. 3. The SINR performance of different receivers with the variety of Fig- 5. The SINR performance of different receivers with the vaiiety of 
cr^/crl for HMT system over DD channel. channel spread factors a^/a^=20dB. 
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where 6(At) in (20) can be rewritten as 



6(A<) = \l- e"^ dx 
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(21) 



Fig. 4. The SINR performance of the proposed Max-SINR receiver for HMT 
system over DD channel. We assume that there is an estimation error between 
the estimated value r4,s and the real value Trms. 



Simulation results show that the proposed scheme is robust 
to the estimation error of RMS and is suitable for the actual 
HMT system. 

Appendix A 
Proof of Equation(19) 

Under the assumption that the prototype pulse at the receiver 
is il}{t) = g{t — At), the SINR of the received signal can be 



where erfc( ) is the complementary error function. If a: > 0, 
we may obtain an approximate solution of the complementary 
error function erfc(-) by |fT9l 



erfc(— ;=) ~ = 

V2 1.64a; + V0.76j:2 + 4 



(22) 



The Max-SINR receiver can be obtained by solving the 
gradient of a(At)&(At) with respect to At 



da(At),,, , d6(At) , 



, da(At) a(At) , db(At) , . 

where A ^ = z — - and }^^' can be expressed as 



dAt 



dAt 

,_(At] 
dAt 



(23) 



db{At) 
dAt 



-At) 



(24) 



hence, equation (23) can be rewritten as 



b{At) 



-At 



1.64 




a 



(25) 



a 27rTrms 



, , /1.527r, a 
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-1 
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Equation (25) can be simplified to a quadratic equation. Under 
the constraint of At > 0, the solution of the quadratic equation 
can be expressed as 



At 



27rr,-r 




3.28Vg 



3.28''g- 



3.52(^)\ (26) 



1.76 



Acknowledgment 

This work was supported by the National Natural Science 
Foundation of China (No. 60972050) and Major Special 
Project of China under Grant (2010ZX03003-003-01) and the 
Jiangsu Province National Science Foundation under Grant 
(BK20 11002) and the Young Scientists Pre-research Fund of 
PLAUST under Grant (No. KYTYZLXY121 1). 



[11] Xu K., Lv Z., Xu Y, Zhang D. Max-SINR Based Timing Synchroniza- 
tion Scheme in Hexagonal Multicarrier Transmission. Wireless Personal 
Communications, Jan. 2012, DOI: 10.1007/s 11 277-0 12-0550-5. 

[12] Jung P., Wunder G. The WSSUS pulse design problem in multicarrier 
transmission. IEEE Transactions on Communications, vol.55, no. 10, 
pp.1918-1928, Oct. 2007. 

[13] Jingxian Wu, Chengshan Xiao, Performance Analysis of Wireless Sys- 
tems With Doubly Selective Rayleigh Fading, IEEE Transactions on 
Vehicular Technology, vol.56, no. 2. pp. 721-730, Mar. 2007. 

[14] Jingxian Wu, Yahong Rosa Zheng, Khaled Ben Letaief, Chengshan 
Xiao, On the EiTor Performance of Wireless Systems with Frequency 
Selective Fading and Receiver Timing Phase Offset, IEEE Transactions 
on Wireless Communications, vol.6, no. 2, pp. 720-729. Feb. 2007. 

[15] Jingxian Wu, Yahong Rosa Zheng, Khaled Ben Letaief, Chengshan Xiao, 
Matched Filter Bound of Wireless Systems over Frequency Selective 
Channels with Receiver Timing Phase Offset, in Proc. IEEE Global 
Telecommun. Conf. GLOBECOM05, pp.3758-3762, Dec. 2005. 

[16] Bello P. A., Characterization of randomly time-variant linear channels, 
IEEE Transactions on Communication System, vol.11, no. 4, pp. 360-393, 
Dec. 1963. 

[17] L. Cohen, Time-frequency analysis. Englewood Cliffs, NJ: Prentice-Hall, 
1995. 

[18] P. Matthias, Mobile fading channels. West Sussex, England: John Wiley 
& Sons, Ltd, 2002. 

[19] Kingsbury, N. Approximation Formulae for the Gaussian Error Integral, 
Q(x), Connexions Web site. http://cnx.Org/content/mll067/2.4/ Jun 7, 
2005. ' 



References 

[1] Kumbasar V., Kucur O. ICI reduction in OFDM systems by using 
improved sine power pulse. Digital Signal Processing, vol.17, no. 6, 
pp.997-1006, Nov. 2007. 

[2] Das S., Schniter P. Max-SINR ISI/ICl-Shaping multicarrier communi- 
cation over the doubly dispersive channel. IEEE Transactions on Signal 
Processing, vol.55, no. 12, pp.5782-5795, Dec. 2007. 

[3] Abbas H.K., Waleed A. M., Nihad S., The perfonnance of multiwavelets 
based OFDM system under different channel conditions. Digital Signal 
Processing, vol.20, no. 2, pp.472-482. Mar. 2010. 

[4] Gao X. Q., Wang W., Xia X. G., et al. Cyclic prefixed OQAM- 
OFDM and its application to single-carrier FDMA. IEEE Transactions 
on Communications, vol.59, no. 5, pp. 1467-1480, May 2011. 

[5] Strohmer T, Beaver S. Optimal OFDM design for time-frequency 
dispersive channels. IEEE Trans. Commun. vol.51, no. 7, pp. 1111-1122, 
Jul. 2003. 

[6] Han F. M., Zhang X. D. Hexagonal multicaiTier modulation: A ro- 
bust transmission scheme for time-frequency dispersive channels. IEEE 
Transactions on Signal Processing, vol.55, no. 5, pp. 1955-1961, May 
2007. 

[7] Han F. M., Zhang X. D. , MLSD for hexagonal multicarrier transmission 
with time-frequency localized pulses, IEEE Transactions on Vehicular 
Technology, vol.58, no.3, pp.1598-1604. Mar 2009. 

[8] Han F. M., Zhang X.D. Asymptotic Equivalence of Two Multicarrier 
Transmission Schemes in Terms of Robustness Against TimeCFre- 
quency Dispersive Channels. IEEE Transactions on Vehicular Technol- 
ogy,\o\.59, no.2, pp.1598-1604, Feb. 2010. 

[9] Xu K., Shen Y. H. Effects of carrier frequency offset, timing offset, 
and channel spread factor on the performance of hexagonal multicarrier 
modulation systems. EURASIP Journal on Wireless Communications 
ami Networking, vol.2009, pp.1-8, Jan. 2009. 
[10] Xu K., Xu Y, Zhang D. SINR analysis of hexagonal multicarrier 
transmission systems in the presence of insufficient synchronization for 
doubly dispersive channel. Frequenz, vol.65, no. 5, pp. 149-157, Aug. 
2011. 



